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ABSTRACT 

The CHALLENGE program was designed to increase 
elementary school children's likelihood of reaching their potential 
in an at-risk environment and, ultimately, to decrease the rate of 
their victimization or social and personal disfunction. The CHALLENGE 
curriculum focuses on increasing children's ability to think, solve 
problems, make effective decisions and choices, communicate, assert 
themselves, manage personal behavior and choices, develop support 
networks, develop self-esteem, take personal responsibility for 
actions and behavior, and set and work toward goals. Two of the four 
schools involved in the CHALLENGE pilot test received a child abuse 
prevention curriculum, the Safe Child Program (SCP) , the previous 
year. Role playing was used to measure behavioral change, and tests 
of knowledge and attitudes, self-esteem, and locus of cot.trol were 
administered. The study sample included 53 boys and 54 girls in 
kindergarten and 1st, 2nd, and 3rd grades at two urban and two 
suburban elementary schools. Control groups consisted of 50 boys and 
40 girls. Treatment groups receiving the CHALLENGE training contained 
children at significantly higher risk than the control groups. The 
treatment groups improved significantly more than the children not 
receiving training, regardless of prior exposure to the SCP. 
CHALLENGE had the greatest impact on the children at greatest risk. 
Teachers were eager to use the program with other children, and 
parents reported improved communication with their children. (AC) 



A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A AAA A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A 



Published: Creating the Quality School: 
University of Oklahoma, May, 



1993 



REACH & CHALLENGE: Evaluating the Effectiveness • %F£^~«^'^TJ£& 
of Programs for At-Risk Youth oeniooi.t.on 0 rooi.cv 

o Sherryll Kraizer, Ph.D., Coalition for Children 

^ Susan S. Witte, M.S.W. , AIDS Related Community Foundation 

g George E Fryer, Jr., Ph.D., University of Colorado 

^ School of Family Medicine 

Q Thomas Miyoshi , Ph.D. Candidate, University of Colorado 

W School of Family Medicine 



In the 30 years since C. Henry Kempe brought child abuse to 
national attention, the magnitude of the problem has become all 
too clear. Prevention, intervention and treatment efforts have 
flourished, but they have not, for the most part, found their way 
into mainstream systems. The most fortunate outcome of focusing 
on the problem of child abuse and its pervasive effects is the 
expanded awareness of the need for prevention of a multitude of 
problems facing youth and families today, inclluding substance 
abuse, teen pregnancy, HIV/AIDS, illiteracy, juvenile delinquency 
and others. At the same time, while we now recognize that there 
are more and more children at varying levels of risk for abuse, 
and that failure to intervene results in expensive and painful 
social and personal dysfunction, we have not yet found a way to 
realistically and effectively address these problems within the 
constrainst of existing resources. 

And the problem has broadened. In addition to the recog- 
nized need for prevention of child sexual, emotional and physical 
abuse, prevention of abduction by strangers and family members 
has been targeted in the past few years. Prevention of substance 
abuse is presently a national priority; and prevention of early 
sexual activity and unwanted pregnancies, as well as the increas- 
ing recognition of AIDS as a risk factor, have served to drama- 
tize the need to delay the onset of sexual activity and to in- 
crease the level of responsibility in sexually active young 
adults . 



What has not been generally recognized to date is the over- 
all inter-relatedness of many of these problems. There are 
direct and well-established links between abuse and feelings of 
^y*t low self-esteem and powerlessness , on the one hand, and between 
V* *) abuse and other forms of social dysfunction, such as substance 
abuse, interpersonal violence, and dropping out of school — to 
name but a few — on the other hand. 
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As understanding has increased about the many areas of risk 
facing young children, the pressure on parents and schools to 
find solutions has reached back-breaking proportions. Only by 
recognizing the overall inter-relatedness of the problems and the 
pervasive role of low self-esteem in virtually all areas of 
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dysfunction can we create viable opportunities to address this 
range of problems in a holistic and powerful way and to engage 
public support by presenting a model which can make a mass of 
problems manageable. If the public and the schools can see a 
realistic and economically viable way to address this problem, it 
is likely that they will take the necessary steps to intervene 
and treat children who may be at risk. 

GENERAL SCIENTIFIC /TECHNOLOGICAL AIMS OF THE PROJECT ; 

Learning is the goal of our educational system; however, 
learning to cope, which is a key factor in the development of 
competence and positive self-image among children, is not so 
clearly part of the curriculum. This is all the more the case 
when we consider how little time is spent in school ensuring that 
children actually can effectively act on their own behalf when 
confronted with interpersonal situations characterized by stress 
or risk. 

The goal of this project is to begin the development and 
evaluation of a two curricu.iua (REACH & CHALLENGE) for use with 
elementary, junior and senior high school children intended to 
reduce their risk to abuse and self -destructive behaviors. Based 
on standardized classroom instruction, with a strong emphasis on 
role-playing as a learning technique which is responsive to the 
multi-faceted risks young people face, the Curriculua objectives 
are to increase the children's ability to demonstrate personal 
safety skills, communication, decision-making and facility in 
calling upon other resources. It is hypothesized that these 
skills, paired with higher self-esteem, decrease their risk to 
abuse, victimization and/or dysfunctional behavior. 

Previous efforts by the authors involved testing The Safe 
Child Program (SCP) specifically targeting prevention of child 
sexual, physical and emotional abuse with children in grades K-3. 
This project provides an opportunity to build on previous efforts 
by broadening the focus in the CHALLENGE Curriculum and evaluat- 
ing its effectiveness. 

The new Program is designed to develop a range of resources 
which will increase the child's likelihood of reaching his or her 
full potential in an at-risk environment and, ultimately, de- 
creasing the rate at which children are victimized and/or exhibit 
social and personal dysfunction. 

The content area skills aim to increase the children's 
ability to: 

* think, problem-solve, make decisions and choices effec- 
tively 

* exercise effective communication and assertiveness 

* manage personal behavior and choices 

* develop and sustain resources for support and nurturance 

* develop and maintain high self-esteem 

* take personal responsibility for actions and choices 

* set goals and work toward them with flexibility 



* prevent child abuse: sexual, physical and emotional 

The hypothesized success of this approach to intervention 
lies in teaching children with innovative and age-appropriate 
techniques , including the use of of roleplay and simulations to 
facilitate actual learning and application of skills. On a more 
longterm basis, this project is the first step in seeking to 
provide the educational community with a basis for considering 
the implementation of a more multi-faceted, cost-effective and 
time-efficient approach to the many risks facing children, rather 
than attempting to prevent each problem singularly. 

The scientific aims of this project included determining: 

1. the overall effect of the curriculum on behavior as 
measured by Roleplay, 

2. the spill-over effect, if any, from previous instruction 
in personal safety skills, 

3. cumulative effects, if any, of exposure to multiple parts 
of the curriculum, and 

4. the relationship of self-esteem (associated with risk in 
previous work) to the ability to learn and demonstrate new 
skills. 

Additional areas of investigation included: 

5. the viability of creating instrumentation for use as a 
"measure of risk" and establishing relationships, if any, between 
risk and the ability to learn and demonstrate new skills, 

6. the relationship of self-esteem/risk to the child's 
learning curve, spill-over effect and cumulative learning, and 

7. the perceived value of the program to parents and teach- 
ers. 

EVALUATION AND RESULTS 

The CHALLENGE Curriculum was written to include the content 
areas cited above, pilot-tested and empirically evaluated using 
four schools, two of which had received the child abuse preven- 
tion curriculum (SCP) in the previous school year. The evaluation 
was based on the classical experimental research design utilizing 
pre- and post-testing of comparison groups. Roleplay was used to 
measure actual behavioral change attributable to the interven- 
tion. (Human subjects approval had been obtained for all proto- 
cols.) In addition, tests of knowledge/attitude, self-esteem and 
locus of control were administered. 

An instrument was created and piloted to assess risk based 
on classroom teacher observations and knowledge. Parents and 
teachers were invited to provide general input throughout the 
Program and to provide an evaluation, personal comments and 
observations upon conclusion of the Program. 

Student samples were selected, in cooperation with school 
officials and parents, to satisfy sample requirements and to 
represent a demographic, age and ethnic cross-section of children 



from those schools. In order to avoid contamination, each group 
was represented by a different school. Actual child participants 
were self-selected to the extent that parents were required to 
agree to their child's participation in the evaluation. 

When the final sample was compiled and the groups were 
compared at pretest, the treatment groups were found to contain 
children who would be considered to be at significantly higher 
risk than those in the control groups. This occurred because 
teachers were aware which would be treatment and which would be 
control schools. Believing that the Program held the possibility 
of making a difference for at risk children, teachers indicated 
that they had made a special effort to get parents of higher risk 
children to participate. Because we were more interested in 
assessing change attributable to the preconditions and the 
Program rather than comparing groups directly, the difference in 
group composition not only did not create a problem, in some ways 
it created a heightened opportunity to match need with benefit. 

Composition of Groups 

Group #1 ; Treatment #1 Received prior prevention (SCP) and CHAL- 
LENGE life skills training 

Control #1 Received prior prevention training (SCP) but no 
CHALLENGE life skills 

Group #2 : Treatment #2 Received no prevention (SCP) training, did 

receive CHALLENGE life skills 

Control #2 Received no training at all 

Description of Sample 



Treatment Control Total 

Rural Schools 112 

Suburban Schools 112 

Boys 53 50 103 

Girls 54 40 94 

Kindergarten 9 16 25 

1st 41 45 86 

2nd 49 23 72 

3rd 8 6 14 

Total participants before attrition 197 



Ethnic breakdown of the sample corresponded with the approxi- 
mate percentages of the total U.S. population. 

This project took place in four schools according to the 
table below: 



Received SCP Pretest CHALLENGE Post-test 

Prevention Training Training 



Treat #1 X 


X 


X 


X 


(n=41) 








Control #1 X 


X 




X 


(n=28 ) 








Treatment #2 


X 


X 


X 


(n=58) 








Control #2 


X 




X 


(n-47) 








Total n=174* 









(* The number of children participating at the conclusion of the 
pretests was 188. This N represents those children who were present 
for all phases of the evaluation. Percentage of attrition was 7.4%) 



ANALYSIS QF F INDINGS 

O BJECTIVE #1: Determine the Overall Effect of the CHALLENGE Life 
Skills Curriculum on Behavior 



Method of Analysis: Compare the change in pre- to post-test 
Roleplay scores of all treatment group children to that of all 
control group children. 



Discussion: 



Pre- to Post-Test Mean Change on Roleplay Scores 







hange 


t-value 


p- value 


Treatment 


#1 (n=41) 


2.37 




p<. 001 


Treatment 


#2 (n-58) 


3.43 




p<.000 


Control # 


1 (n=28) 


.79 




p<.145 


Control # 


2 (n=47) 


.62 




p<.014 


Treatment 


#1 & #2 (n=99) 


2.99 


-5.52 




Control 


#1 & #2 (n=75) 


.68 







This difference demonstrates that the treatment groups, 
receiving the CHALLENGE training, improved significantly more 
than the children not receiving training, regardless of prior 
exposure to the SCP child abuse prevention training. This dif- 
ference in performance is significant at p<.001, which supports 
the hypothesis that with participation in the CCHALLENGE Program 
children can and do learn new behavioral skills which are at- 
tributable to the programming. 

The significance of this change in behavior lies in the 
child's demonstration, through the Roleplay, of the ability and 
willingness to speak up in the face of discomfort, bribery, 
coercion and intimidation. This behavioral change embodies 
communication, assertiveness , decision-making and self-esteem, 
all of which are hypothesized to be associated with reduction of 
risk to abuse and personal dysfunction. 
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OBJECTIVE #2: Determine the Spill-Over Ef fect of Previous SCP 
Curriculum on Behavior 



Merhod of Analysis: Compare the pretest Roleplay scores of all 
children in treatment and control groups #1 (those who had previ- 
ous prevention program) to those of all children in treatment and 
control groups #2 (those who did not have previous training) . 



Discussion: 



In order to be clear about the singular or combined effects 
of the SCP and the new CHALLENGE Program in terms of behavioral 
change, it is important tc analyze the pretest scores of Group 
#1, who had previous prevention training, and Group #2, who did 
not, as in the table below: 

Mean Scores to Determine Spill-Over Effect 

Group #1 Group #2 

Treat. #1 Cont. #1 Combined Treat. #2 Cont. #2 Combined 
Pretest 5.37 6.61 5.99 3.86 3.40 3.63 

Simulation 

The difference in pretest scores for those children who had 
previous SCP training indicates that they began with a higher 
demonstrated skill level than those with no previous training. 
This will be important in the analysis of cumulative effects 
which follows. 

OBJECTIVE #3: Determine the Cumulative Effect of CHALLENGE cur- 
riculum on Behavior 

Method of Analysis : Compare the change in pre- to post-test 
Roleplay scores of children in Treatment #1 (received prior 
training) to that of Treatment #2 (received no prior training). 



Discussion: 



Cumulative effects of the CHALLENGE Curriculum on behavior 
was determined by comparing changes from pre- to post-test on the 
Simulation/Roleplay for children in the two treatment groups. In 
treatment groun #i, which received both the SCP and CHALLENGE 
training, actual change attributable to prevention and life 
skills programming = 3.93, compared with treatment group #2, 
where change = 3.43. 

Cumulative Effects of Program ( s ) on Behavior 



SCP Pretest 
(last year) 

Treatment #1 3.80 

(received SCP) 

Treatment #2 

(no prior program) 



CHALLENGE Pretest 
(SBIR Phase 1) 
5.37 

3.86 



Post-Test Actual 
Change 
7.73 3.93** 

7.29 3.43** 

** p<.001 
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This finding supports the hypothesis that the CHALLENGE 
training, in addition to the SCP, maximizes the value to the 
children, although it must also be considered a possibility that 
the higher score of Treatment #1 is attributable to the redose 
effect, that is, having programming a second time. (This effect, 
while different, would also be highly desirable in a Curriculum 
which is presented year after year as is the plan for the Curric- 
ulum being developed,) 

OBJECTIVE Ml Determine the Role of Self-Esteem 

Method of Analysis: Repeat analyses 1-3 substituting self-esteem 
scores for Simulation/Roleplay scores. Then, repeat analysis 1 
controlling for self-esteem score 



Mean Change on Self-Esteem Scores Pre- to Post-Test 

Change T-value p-value 

Treatment #1 & #2 .87 
(received life skills) 
Control # 1 & #2 .19 
(did not receive life skills) 

1.32 .19 



Mean Scores to Determine Spill -Over Effect of Self-Esteem 

Group #1 Group #2 

Treat #1 Cont #1 Combined Treat #2 Cont #2 Combined 

Pretest 22.50 24.00 23.25 20. 60 21.34 20.97 

Self-esteem 

The pretest scores of Group #1 (received prior SCP) reflects 

higher self-esteem scores which were, in the previous work, 
attributable to the intervention. 

Cumulative Effects of Program(s) on Self-Esteem 

Actual PreTest Life Skills Pretest Post-test Actual 
(last year) (Phase 1) Change 

Treat #1 21.30 22.5 21.97 .67 

(received prior program) 

Treat #2 20.6 21.3 .71 

(no prior program) 

Discussion; 

Changes in self-esteem scores have been closely associated 
with risk in previous efforts. While changes with this instru- 
mentation were not significant for any groups in this pilot 
project, the role of risk (which had been the principal purpose 
of using the self-esteem instrumentation in past efforts) was 
measured with new instrumentation developed as a part of this 
project. 
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MEASURE OF RISK 



The "measure of risk" instrument created and piloted in this 
project includes items concerning classroom performance and 
attitude such as: disruptive in class, withdrawn, fearful, de- 
pendent, poor concentration, poor hygiene, specific academic 
problems, and language deficits. Other known factors from the 
child's environment such as child abuse, death in the family, 
malnutrition, etc., are also rated. Each of the items received a 
rating of 0=no problem, 1= mild problem, 2=serious problem and 
3=very serious problem. The total of the ratings of the class- 
room teacher established the risk score. The risk scores for all 
children ranged from 1 to 76. 

Eguating the pretest risk scores with pretest levels of 
self-esteem indicated that there is a direct correlation between 
the mean level of risk and the mean level of self-esteem for each 
school factoring in the effect of prior programming on self- 
esteem scores coming into this project. Within groups, those who 
had the prior SCCP and those who did not, the relationship of 
self-esteem to risk held. That is to say, those children with 
the lowest level of self-esteem had the highest risk scores and 
those who had previously participated in the SCP evidenced less 
risk. 



Shown below are the mean risk scores for each school com- 
pared to change scores on the roleplay. These groups were not 
matched and the differences in risk reflect the incoming nature 
of the group. As explained earlier, the higher level of risk in 
the treatment schools is attributable to the teachers encourage- 
ment of parents of known at risk children to have their children 
participate. What should be noted is the degree of benefit in the 
program based on the degree of incoming risk. 

Mean Risk tQ Mean Change Scores (Treatment Groups Only) 

Mean Risk Score • Mean Change Score 
Suburban 1 (N= 28) 2.0 .79 

Suburban 2 (N=47) 2.5 .62 

Rural (N=41) 8.0 2.37 

Inner City (N=58) 14.0 3.43 

Most simply stated, this table tells us that the programming 
provided to the children had the greatest impact for those chil- 
dren at the greatest risk. The importance of this cannot be 
stated too strongly. Allocation of resources must reflect atten- 
uation of need. That is to say, intervention only makes a real 
difference when risk is present. Given the level of problems we 
are trying to prevent, it is not a good use of resources to 
provide service where the provision of service makes no substan- 
tive difference. 



ADDITIONAL FINDINGS: 



This project included a qualitative evaluation component . 
Parents (33) and teachers (12) were informally interviewed 
(usually before and after the last few sessions) and given an 
opportunity (through a written survey) to provide feedback on 
the Program and its effect on the participating children. Teach- 
ers were consistently supportive and eager to have the Program 
available for use with ali of their children. They felt they 
could use the Program with minimal training (2-3 hours). Their 
subjective reports were strongly supportive of the empirical 
evidence presented above. 

This is a significant finding because these teachers are the 
very people who will be called upon to implement the program from 
year to year. Their support and perception that it will benefit 
the children they believe to be most at risk cannot be underval- 
ued. For example, two teachers reported that the children in 
their rooms began to build a network of support for those indi- 
viduals most at risk. This peer support resulted in greater 
opportunities for positive experiences for the at-risk child, a 
corresponding gain in self-esteem, and an increased ability to be 
successful in classroom and learning situations. This, in turn, 
generated teacher enthusiasm and a greater sense of "making a 
difference. 11 

Parents reported a consistently higher level of communica- 
tion with their children, particularly in their children's will- 
ingness to express feelings and to discuss problems. They also 
reported that their children took more responsibility for them- 
selves at home. This was reflected in simple, but significant, 
activities such as making their beds, cleaning their rooms, 
packing their own lunches for school and more appropriate and 
supportive interactions with siblings. 

SUMMARY 

The results of this pilot project, with the introduction of 
a new measure for assessment of risk and the CHALLENGE Program 
which combines personal safety training with broader behavioral 
and attitudinal skills, tells us clearly that we can make a 
measurable and statistically significant difference for all 
children, but, most importantly, those children at the greatest 
risk. 

As indicated in the introduction, there is a well estab- 
lished link between abuse and feelings of low self-esteem and 
powerl essness , on the one hand, and between abuse and other forms 
of social dysfunction, like substance abuse, interpersonal vio- 
lence and dropping out of school — to name but a few — on the 
other hand^ 

This pilot project has begun the process of creating a 
school-based curriculum and demonstrated it's ability to change 
children's behavior in a positive direction — directly associat- 
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ed with reducing risk to abuse — and to increase children's 
ability to demonstrate important related skills such as asser- 
tiveness, communication, decision-making and asking for help. It 
has laid the groundwork for the refinement of instrumentation 
which measures risk and value received from intervention. 

These preliminary results direct us to further develop the 
Curriculum and to refine the evaluation techniques toward the end 
of making effective prevention and life skills training accesssi- 
ble for those children most at risk. 
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